newt

package module
v0.0.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 3, 2025 License: BSD-3-Clause Imports: 24 Imported by: 0

README

Newt Project

Newt is an experimental rapid application development. Specifically Newt is focused on creating web based metadata curation tools. These types of applications are commonly needed in galleries, libraries, archives and museums (abbr: GLAM). Newt makes creating these type of applications easier.

How does Newt do that? Newt generates an application by implementing a service oriented architecture using a combination of off the shelf software and generated code.

You can think of a web application as a sequence of requests and responses. Traditionally a web browser contacts your web site or application then one of two things will happen. Your app knows the answer and hands back the result. Alternatively if it doesn't know the answer it tells you it can't do so (e.g. 404 HTTP STATUS CODE). With service oriented architecture your application has another option. Your application can contact another service and use that result to answer the request from the web browser. Newt's implements a service oriented architecture by orchestrating data processing pipelines[^11].

[^11]: A data pipeline is formed by taking the results from one web service and using it as the input to another web service. It is the web equivalent of Unix pipes. Prior art: Yahoo! Pipes

With data pipelines we can accept a request and feed that request to one service then take its output and send it to the next service. Newt does this by providing a data router. Newt can manage the request sequence through a simple YAML described pipeline. While it is possible to create pipelines using Apache and NginX proxy features in practice that approach quickly becomes an unmanageable configuration problem. You could encapsulate clusters of processes in containers but this too becomes complex to manage. Newt's router can cut through the hairball of configurations and define pipelines per request route. With Newt's pipelines the last service completed hands its result back to Newt's router which returns the result to the web browser.

The data pipelines are defined in Newt' YAML configuration file. The pipelines are managed by Newt's data router.

Why is this important? Much of the "back end" of a web application is already available as off the shelf software. Here is a short list of examples used by Caltech Library.

This is not an exhaustive list. These types of applications can all be integrated into your application through configuring the connection in Newt's YAML file. Newt's router runs the data pipelines and can host static content.

Wait, what about my custom metadata needs?

Metadata oriented applications share the following operations -- create, retrieve, update, delete and list. These are called CRUD-L[^12] features. Customization tend towards data models. Newt's configuration file includes simple YAML description of your data models. It uses the data model to render configuration, middleware and templates. Newt's data router ties them all together into an application using service oriented architecture.

Newt provides:

  • a simple data modeler
  • code generation
  • template engine
  • data routing

You can extend your application through browser side enhancements, adding additional data routes and pipelines or through customizing the generated code.

[^12]: CRUD-L, acronym meaning, "Create, Read, Update, Delete and List" or alternately "Create, Retrieve, Update, Delete and List". These are the basic actions used to manage metadata.

Does Newt clean my house or make cocktails?

Newt is a narrowly focused rapid application development toolbox. Newt will not clean your house or make a cocktail. Additionally it does not support the class of web applications that handle file uploads. That means it is not a replacement for Drupal, WordPress, Islandora, etc. Newt is for building applications more in line with ArchivesSpace but with simpler data models. If you need file upload support Newt is not the right solution at this time.

Newt applications are well suited to interacting with other applications that provide a JSON API. A service with a JSON API can be treated as a JSON data source. A JSON data source can easily be run through a pipeline. Many GLAM applications like ArchivesSpace and Invenio RDM provide JSON API. It is possible to extended those systems by creating simpler services that can talk to those JSON data sources. Newt is well suited to this "development at the edges" approach.

What if those systems aren't available on localhost?

Services not available on localhost must be proxied to integrate with your Newt application. Why is this necessary? Short answer, security. Newt seeks to reduce the attack surface of your web application as much as possible. It does that by only trusting services offered directly on localhost. Newt's application models presumes you're running behind a "front end web server" like Apache 2, NginX or Lighttd. These systems can be configured to provide access control as well as perform proxy services. The front end web server is your first line of defense against a cracker.

External web services integrate through a proxy setup (e.g ORCID, ROR, CrossRef or DataCite). This can be done via the front end web server or by writing a dedicated proxy service. Today writing proxy services us easily accomplished in most popular programming languages due to good support for web protocols. This is true of Python, PHP, JavaScript, Go, Rust and many others.

How does Newt impact web application development?

A Newt application encourages the following.

  • modeling your data simply the result is expressed in YAML
  • preference for "off the shelf" over writing new code
  • use a database management system for managing your data and SQL functions
  • prefer software that can function as a JSON data source
  • transforming data representations by using a light weight template engine
  • code generation where appropriate

If Newt doesn't make cocktails, what is it bringing to the party?

In 2024 there is allot of off the self software to build on. Newt provides a few tools to fill in the gaps.

  • newt is a development tool for data modeling, generating code, running Newt router, template engine and PostgREST or dataset collections
  • ndr is a stateless web service (a.k.a. micro service) that routes a web requests through a data pipelines built from other web services
  • nte is a stateless template engine inspired by Pandoc server that supports the Handlebarsjs template language and is designed to process data from a JSON data source

The Newt YAML configuration ties these together expressing

  • data modeling (descriptions of data as you would provided in a web form)
  • code generation
  • data routes (web requests differentiated by a HTTP method and URL path that trigger processing in a data pipeline)
  • template maps (path/template pairs used that can recieve JSON and render a results)

What type of applications are supported by Newt?

Most GLAM applications are focused on managing and curating some sort of metadata records. Sometimes these metadata records are quite complex (e.g. ArchivesSpace, RDM records) but often they are simple (e.g. a list of authors, a list of citations). Newt's primary target is generating applications to manage simple data models. Simple data models are those which can be expressed in web forms via HTML5 native input elements.

Motivation

Over the last several decades web applications became very complex. This complexity is expensive in terms of reliability, enhancement, bug fixes and software sustainability. Newt is an attempt to address this by reducing the code you write and focusing your efforts on declaring what you want.

In 2024 the back end of web applications can largely be assemble from off the shelf software. Middleware however remains complex. I believe this to be a by product of inertia in software development practices and the assumption that what is good for "Google Scale" is good for everyone.

I think a radical simplification is due. Newt is intended to spark that conversation. My observation is most software doesn't need to scale large. In the research and GLAM communities we don't routinely write software that scales as large as Zenodo. We don't typically support tens of thousands of simultaneous users. I think we can focus our efforts around orchestrating off the shelf components and put our remaining development time into improving the human experience using our software. A better human experience is an intended side effect of Newt.

A big key to simplification is narrowing the focus of our middleware. When our middleware has to implement everything it becomes very complex. Look at Drupal and WordPress. They implement data modeling, data management, user accounts, access management, data transformation.

I think our web services should be doing less, much less. Our web services should be narrowly focused. Conceptually simpler. Do one or two things really well. Newt enables using simpler discrete services to build our applications.

Working with off the shelf deliverables

Take the following as a "for instance".

  • (data management) Postgres combined with PostgREST gives you an out of the box JSON API for managing data
  • (access control) Apache 2 or NGINX combined with Shibboleth for access control and communicating with the web browser
  • (rich client) Web browsers now provide a rich software platform in their own right
  • (optionally you can full text search) Solr gives you a powerful, friendly, JSON API for search and discovery

With the above list we can build capable applications relying on the sophisticated features of our web browsers. This is true even without using Newt. There is a problem though. If we only use the above software to build our application we must rely on JavaScript (or WASM module) running in the web browser to interact with the server. This sounds simple. In practice this is a terrible idea[^17].

[^17]: See https://infrequently.org/2024/01/performance-inequality-gap-2024/ for a nice discussion of the problem.

What we should do is use Newt to tie those JSON services together and send rendered HTML back to the web browser. Newt's router provides static file service and a means of pipelining our JSON data source through a template engine. Newt provides a Handlebars[^18] template engine for that purpose. Newt provides the missing bits from my original list so we don't need to send JavaScript down the wire to the web browser. The Newt approach uses less bandwidth, fewer network accesses and less computations cycles on your viewing device. The Newt approach takes advantage of what the web browser is really good at without turning your web pages into a web service. Newt YAML describes the system you want. You get the Newt capabilities without writing much code. Maybe without writing any code if Newt's code generator does a sufficient job for your needs.

[^18]: Handlebars is largely a superset of the Handlbars Template Language, see https://handlebarsjs.com and https://mustache.github.io for details.

A "newt" baseline

Web services talk to other web services all the time. This isn't new. It isn't exotic. Newt scales down this approach to the single application.

  • Can we align access control with our front end web server?
  • Can we insist on our database management system providing a JSON API?
  • Can we treat the output of one web service as the input for the next?
  • Can we aggregate these into data pipelines?
  • Will that be enough to define our web application?

In Spring 2024 for metadata curation apps I think the answer is "yes we can".

What comes with the Newt Project?

The primary tools are.

  • newt a developer tool for building a Newt based application which includes modeling, code generation support, and runtime support
  • ndr a web service designed for working with other "off the shelf" web services. It functions both as a router and as a static file server. It does this by routing your request through a YAML defined pipeline and returning the results. Typically this will be a JSON data source and running that output through a template engine like Newt's nte.
  • nte provides a Handlebars template engine for transforming JSON data into a more human friendly format.

See the user manual for details.

About the Newt source repository

Newt is a project of Caltech Library's Digital Library Development group. It is hosted on GitHub at https://github.com/caltechlibrary/newt. If you have questions, problems or concerns regarding Newt you can use GitHub issue tracker to communicate with the development team. It is located at https://github.com/caltechlibrary/newt/issues. The name comes from wanting a "[New t]ake" on web application development.

Getting help

The Newt Project is an experiment!!. The source code for the project is supplied "as is". Newt is a partially implemented prototype (May 2024). However if you'd like to ask a question or have something you'd like to contribute please feel free to file a GitHub issue, see https://github.com/caltechlibrary/newt/issues.

Currently Newt is targeted at Windows on X86, macOS on x86 and ARM (e.g. M1, M2) and Linux on aarch64 (ARM 64) and x86.

Building from source

Newt is experimental so doesn't have installers yet. If you are compiling from source the following software is required.

  1. Git to retrieve the Newt repository
  2. Golang >= 1.22 (used to compile command line programs and services except newthandlebars)
  3. Deno >= 1.44 (used to build newthandlebars, note not available on Windows ARM or 32bit Linux)
  4. Pandoc >= 3.1 (used to build version.go, version.ts and documentation)
  5. GNU Make and Bash shell (you can cross compile for Windows and macOS from Linux)

Steps to compile

  1. From your home directory clone the repository
  2. change into the repository directory
  3. Run Make
  4. Run make test
  5. Run make install
cd $HOME
git clone [email protected]:caltechlibrary/newt src/github.com/caltechlibrary/newt
cd  src/github.com/caltechlibrary/newt
make
make test
make install

Documentation

Overview

*

  • ast.go holds the data structure that defines Newt applications. *
  • @author R. S. Doiel

*

  • cli.go an implements runners for the cli of the Newt Project. *
  • @author R. S. Doiel

Index

Constants

View Source 
const (
	// These constants are used for exit code. FIXME: look up the POSIX recommendation on exit
	// codes and adopt those.
	OK = iota
	CONFIG

	// General failure of a command or service
	INIT_FAIL
	CHECK_FAIL
	MODELER_FAIL
	GENERATOR_FAIL
	BUILD_FAIL
	ROUTER_FAIL
	TEMPLATE_ENGINE_FAIL
	NEWT_FAIL
	SWS_FAIL
	POSTGREST_FAIL

	// Internal service failures
	RESOLVE
	HANDLER
	SERVER_ERROR
	UNSUPPORTED_ACTION
	DATA_ERROR
	READ_ERROR
	DECODE_ERROR
	TEMPLATE_ERROR

	// Default service settings
	ROUTER_PORT                  = 8010
	TEMPLATE_ENGINE_PORT         = 8011
	TEMPLATE_ENGINE_TIMEOUT      = 3 * time.Second
	TEMPLATE_ENGINE_BASE_DIR     = "views"
	TEMPLATE_ENGINE_PARTIALS_DIR = "partials"
	TEMPLATE_ENGINE_EXT_NAME     = ".hbs"
	SWS_PORT                     = 8000
	SWS_HTDOCS                   = "."
	POSTGREST_PORT               = 3000
	POSTGRES_PORT                = 5432
)
View Source 
const (
	TuiStandardMenuHelp = `Enter menu letter and id`
	TuiStandardMenu     = `Menu [a]dd, [m]odify, [r]emove or press enter when done`
)
View Source 
const (
	// Version number of release
	Version = "0.0.9"

	// ReleaseDate, the date version.go was generated
	ReleaseDate = "2025-04-02"

	// ReleaseHash, the Git hash when version.go was generated
	ReleaseHash = "3429ddf"

	LicenseText = `` /* 1430-byte string literal not displayed */

)

Variables

This section is empty.

Functions

func FmtHelp 

func FmtHelp(src string, appName string, version string, releaseDate string, releaseHash string) string

FmtHelp lets you process a text block with simple curly brace markup.

func NewtStaticFileServer 

func NewtStaticFileServer(port int, htdocs string, verbose bool) error

func PatternKeys 

func PatternKeys(p string) []string

PatternKeys parses a pattern and returns a list of keys found. NOTE: this could be improved to make sure that delimiters are paired and that the pattern's names do not contain spaces.

func RunBuilder added in v0.0.9 

func RunBuilder(in io.Reader, out io.Writer, eout io.Writer, args []string) int

RunBuilder is a runner for take the generating SQL, templates, etc. and Generate the validator middleware a well as updating the Postgres+PostgRESTS configuration and databases.

func RunGenerator added in v0.0.9 

func RunGenerator(in io.Reader, out io.Writer, eout io.Writer, args []string) int

RunGenerator is a runner for generating SQL and templates from our Newt YAML file.

func RunModeler added in v0.0.9 

func RunModeler(in io.Reader, out io.Writer, eout io.Writer, args []string) int

func RunNewt 

func RunNewt(in io.Reader, out io.Writer, eout io.Writer, args []string, verbose bool) int

RunNewt is a runner that can run Newt's router and template engine plus PostgREST if defined in the Newt YAML file.

func RunNewtCheckYAML 

func RunNewtCheckYAML(in io.Reader, out io.Writer, eout io.Writer, args []string, verbose bool) int

NewtRunCheckYAML will load a Newt YAML fiel and make sure it can parse the configuration.

func RunNewtConfig added in v0.0.9 

func RunNewtConfig(in io.Reader, out io.Writer, eout io.Writer, args []string, verbose bool) int

RunNewtConfig will initialize a Newt project by creating a Newt YAML file interactively.

func RunNewtServices added in v0.0.9 

func RunNewtServices(in io.Reader, out io.Writer, eout io.Writer, args []string, verbose bool) int

RunNewtServices will run the applictions defined in your Newt YAML file.

func RunRouter added in v0.0.9 

func RunRouter(in io.Reader, out io.Writer, eout io.Writer, args []string, dryRun bool, port int, htdocs string, verbose bool) int

RunRouter is a runner for Newt data router and static file service

func RunStaticWebServer 

func RunStaticWebServer(in io.Reader, out io.Writer, eout io.Writer, args []string, port int, verbose bool) int

RunStaticWebServer this provides a localhost for static file content.

func RunTemplateEngine added in v0.0.9 

func RunTemplateEngine(in io.Reader, out io.Writer, eout io.Writer, args []string, port int, timeout int, verbose bool) int

RunTemplateEngine is a runner for a Newt's template engine.

func TmplFooterPartial added in v0.0.9 

func TmplFooterPartial(out io.Writer, copyrightYear string, copyrightLink string, copyrightText string, licenseLink string, licenseText string, contactAddress string, contactEMail string, contactPhone string) error

func TmplGen added in v0.0.9 

func TmplGen(out io.Writer, model *models.Model, action string) error

TmplGen takes an io.Writer, model and an action string rendering the contents of the model as a Newt handlebars template for the provided action. It returns an error value when something goes wrong.

func TmplHeadPartial added in v0.0.9 

func TmplHeadPartial(out io.Writer, defaultTitle string, cssPath string) error

func TmplHeaderPartial added in v0.0.9 

func TmplHeaderPartial(out io.Writer, defaultHeaderLink string, defaultHeaderLinkTitle string, defaultLogoLink string, defaultLogoTitle string) error

func TmplNavPartial added in v0.0.9 

func TmplNavPartial(out io.Writer, navElement string) error

func UnmarshalAST added in v0.0.9 

func UnmarshalAST(src []byte, ast *AST) error

UnmarshalAST will read []byte of YAML or JSON, populate the provided *AST object and return an error.

``` src, _ := os.ReadFile("app.yaml") ast := new(AST) 

if err := UnmarshalAST(src, ast); err != nil {
    // ... handle error
}

```

Types

type AST added in v0.0.9 

type AST struct {
	// AppMetadata holds your application's metadata such as needed to render an "about" page in your final app.
	AppMetadata *AppMetadata `json:"app_metadata,omitempty" yaml:"app_metadata,omitempty"`

	// Services holds definitions of the services used to compose your application.
	// and enough metadata to generated appropriate Systemd and Luanchd configurations.
	Services []*Service `json:"services,omitempty" yaml:"services,omitempty"`

	// Models holds a list of data models. It is used by
	// both the data router and code generator.
	Models []*models.Model `json:"models,omitempty" yaml:"models,omitempty"`

	// Routes holds an array of maps of route definitions used by
	// the data router and code generator
	Routes []*Route `json:"routes,omitempty" yaml:"routes,omitempty"`

	// Templates holds an array of maps the request to template to request for
	// Newt (Handlebars) template engine
	Templates []*Template `json:"templates,omitempty" yaml:"templates,omitempty"`
	// contains filtered or unexported fields
}

AST holds a configuration for Newt for the data router and code generator.

func LoadAST added in v0.0.9 

func LoadAST(configFName string) (*AST, error)

LoadAST read a YAML file, merges environment variables and returns a AST object and error value.

``` ast, err := LoadAST("app.yaml") 

if err != nil {
    // ... handle error
}

```

func NewAST added in v0.0.9 

func NewAST() *AST

NewAST will create an empty AST with top level attributes

func (*AST) AddModel added in v0.0.9 

func (ast *AST) AddModel(model *models.Model) error

AddModel takes a new Model, checks if the model exists in the list (i.e. has an existing model id that matches the new model and if not appends it so the list.

func (*AST) Check added in v0.0.9 

func (ast *AST) Check(buf io.Writer) bool

Check reviews the ast *AST and reports and issues, return true if no errors found and false otherwise. The "buf" will hold the error output.

func (*AST) Encode added in v0.0.9 

func (ast *AST) Encode() ([]byte, error)

func (*AST) GetAllTemplates added in v0.0.9 

func (ast *AST) GetAllTemplates() []string

GetAllTemplates returns a list of templates, including partials defined in the .Templates property. Part template names are indented with a "\t"

func (*AST) GetModelById added in v0.0.9 

func (ast *AST) GetModelById(id string) (*models.Model, bool)

GetModelById return a specific model by it's id

func (*AST) GetModelIds added in v0.0.9 

func (ast *AST) GetModelIds() []string

GetModelIds returns a list of model ids

func (*AST) GetPrimaryTemplates added in v0.0.9 

func (ast *AST) GetPrimaryTemplates() []string

GetPrimaryTemplates return a list of primary template filenames

func (*AST) GetRouteIds added in v0.0.9 

func (ast *AST) GetRouteIds() []string

GetRouteIds returns a list of Router ids found in ast.Routes

func (*AST) GetService added in v0.0.9 

func (ast *AST) GetService(appName string) *Service

GetService takes a list of applications, `[]*Service`, and returns the application name in the list or nil.

func (*AST) GetTemplateByPrimary added in v0.0.9 

func (ast *AST) GetTemplateByPrimary(fName string) (*Template, bool)

GetTemplateByPrimary returns the template entry using primary template filename

func (*AST) GetTemplateIds added in v0.0.9 

func (ast *AST) GetTemplateIds() []string

GetTemplateIds return a list of template ids.

func (*AST) HasChanges added in v0.0.9 

func (ast *AST) HasChanges() bool

func (*AST) RemoveModelById added in v0.0.9 

func (ast *AST) RemoveModelById(id string) error

RemoveModelById find the model with the model id and remove it

func (*AST) RemoveRouteById added in v0.0.9 

func (ast *AST) RemoveRouteById(id string) error

RemoveRouteById find the route with route id and remove it

func (*AST) RemoveService added in v0.0.9 

func (ast *AST) RemoveService(appName string) error

RemoveService takes a list of applications, `[]*Service`, and remove the target item.

func (*AST) RemoveTemplateById added in v0.0.9 

func (ast *AST) RemoveTemplateById(id string) error

RemoveTemplateById() find the template id and remove it from the .Templates structure

func (*AST) SaveAs added in v0.0.9 

func (ast *AST) SaveAs(configName string) error

SaveAs writes the *AST to a YAML file.

func (*AST) UpdateModel added in v0.0.9 

func (ast *AST) UpdateModel(id string, model *models.Model) error

UpdateModel takes a model id and new model struct replacing the existing one.

type AppMetadata added in v0.0.9 

type AppMetadata struct {
	AppName        string `json:"name,omitempty" yaml:"app_name,omitempty"`
	AppTitle       string `json:"title,omitempty" yaml:"app_title,omitempty"`
	CopyrightYear  string `json:"copyright_year,omitempty" yaml:"copyright_year,omitempty"`
	CopyrightLink  string `json:"copyright_link,omitempty" yaml:"copyright_link,omitempty"`
	CopyrightText  string `json:"copyright_text,omitempty" yaml:"copyright_text,omitempty"`
	LogoLink       string `json:"logo_link,omitempty" yaml:"logo_link,omitempty"`
	LogoText       string `json:"logo_text,omitempty" yaml:"logo_text,omitempty"`
	LicenseLink    string `json:"license_link,omitempty" yaml:"license_link,omitempty"`
	LicenseText    string `json:"license_text,omitempty" yaml:"license_text,omitempty"`
	CSSPath        string `json:"css_path,omitempty" yaml:"css_path,omitempty"`
	HeaderLink     string `json:"header_link,omitempty" yaml:"header_link,omitempty"`
	HeaderText     string `json:"header_text,omitempty" yaml:"header_text,omitempty"`
	ContactAddress string `json:"contact_address,omitempty" yaml:"contact_address,omitempty"`
	ContactPhone   string `json:"contact_phone,omitempty" yaml:"contact_phone,omitempty"`
	ContactEMail   string `json:"contact_email,omitempty" yaml:"contact_email,omitempty"`
}

AppMetadata holds metadata about your Newt Service This is primarily used in generated Handlbars partials

type BodyGen added in v0.0.9 

type BodyGen func(io.Writer, *models.Model) error

type Generator added in v0.0.9 

type Generator struct {
	// Namespace is used when generating the SQL/conf for setting up Postgres+PostgREST
	Namespace string

	// Models holds the models used to generator specific code
	Models []*models.Model

	// Options holds the result environment variables and options that can be used in generator code
	Options map[string]interface{}

	// AppMetadata holds the metadata for the application being generated
	AppMetadata *AppMetadata

	// Postgres configuration information
	Postgres *Service

	// PostgREST configuration information
	PostgREST *Service

	// TemplateEngine configuration information
	TemplateEngine *Service
	// contains filtered or unexported fields
}

Generator holds our Newt Generator structures for rendering code.

func NewGenerator 

func NewGenerator(ast *AST) (*Generator, error)

NewGenerator instaitates a new Generator object form a filename and AST object It returns a Generator object and error value.

func (*Generator) Generate added in v0.0.9 

func (g *Generator) Generate(generatorName string, modelId string, action string) error

Generator generates the code based on the contents of Generator struct. It will also verify that the needed parameters are provided.

- generatorName is the generator to use - action is a parameter that the selected generator can use (e.g. PostgreSQL has setup as well as ) - modelId references the `.id` attribute of the model needing code generation

type Logger 

type Logger struct {

	// After (defaults to true) logs the request after running the wrapped handler
	After bool

	// Verbose (defaults to false) show the contents of a GET, POST, PUT and DELETE in log output
	Verbose bool
	// contains filtered or unexported fields
}

Logger implementes Newt Project's web logging

func NewLogger 

func NewLogger(handlerToWrap http.Handler) *Logger

NewLogger constructs a new Logger middleware handler

func (*Logger) ServeHTTP 

func (l *Logger) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP handles the request by passing it to the real handler and logging the request details

type Route added in v0.0.9 

type Route struct {
	// Id holds an id for the route. It is unique for each set of HTTP methods. E.g. The id can
	// tie a GET (retrieve a webform) and POST (submit the form) but NOT two GET or two POST.
	// This allows us to work with simple and complex actions.
	Id string `json:"id,omitempty" yaml:"id,omitempty"`

	// Pattern holds the HTTP Method and URL path, may include Go 1.22 patterns
	Pattern string `json:"request,required" yaml:"request,omitempty"`

	// Description holds a human describe of the purpose of this route.
	Description string `json:"description,omitempty" yaml:"description,omitempty"`

	// Pipeline holds the series of http services context with the output of one sent to another.
	Pipeline []*RouteService `json:"pipeline,omitempty" yaml:"pipeline,omitempty"`

	// Debug if true log verbosely
	Debug bool `json:"debug,omitempty" yaml:"debug,omitempty"`

	// Env holds a map of defaults that are available from the environment and from path values in the url
	Options map[string]interface{} `json:"options,omitempty" yaml:"options,omitempty"`

	// Vars holds the variables defined in the route
	Vars []string `json:"-" yaml:"-"`
}

This holds the route definitions, e.g. request, description, pipeline, debug

func (*Route) Check added in v0.0.9 

func (nr *Route) Check(buf io.Writer) bool

Check will review the route and return true if no errors and false othise. It will write error descriptions to buf.

func (*Route) Handler added in v0.0.9 

func (nr *Route) Handler(w http.ResponseWriter, r *http.Request)

Handler creates an http handler func for a given route.

func (*Route) ResolvePattern added in v0.0.9 

func (nr *Route) ResolvePattern(r *http.Request) map[string]interface{}

ResolvePattern takes the request Pattern and pulls out the values from the actual request. returns values in map[string]string and an error value.

func (*Route) ResolveRoute added in v0.0.9 

func (nr *Route) ResolveRoute() error

ResolveRoute reviews the `.Request` attribute and updates the Vars using PatternKeys()

func (*Route) RunPipeline added in v0.0.9 

func (nr *Route) RunPipeline(w http.ResponseWriter, r *http.Request, env map[string]interface{}) ([]byte, int, string, error)

type RouteService added in v0.0.9 

type RouteService struct {
	// Service holds the http Request Pattern to request a reast from a service
	Service string `json:"service,required" yaml:"service,omitempty"`

	// Description describes the service and purpose of contact. Human readable.
	Description string `json:"description,omitempty" yaml:"description,omitempty"`

	// Timeout sets a timeout value to recieve a response from the service.
	Timeout time.Duration `json:"timeout,omitempty" yaml:"timeout,omitempty"`
}

RouteService holds the necessary information to contact a data ast and retrieve the results for use in a pipeline.

func (*RouteService) MakeRequest added in v0.0.9 

func (s *RouteService) MakeRequest(env map[string]interface{}, input []byte, contentType string, debug bool) ([]byte, int, string, error)

Request a service, sending any import if provided. Returns a byte splice of results and an error value

type Router added in v0.0.9 

type Router struct {
	// Port is the port the router will listen on
	Port int

	// Routes holds a list of route
	Routes []*Route

	// Htdocs holds the location of a the static files if used
	Htdocs string
}

Router is used to implement the Newt Router

func NewRouter added in v0.0.9 

func NewRouter(ast *AST) (*Router, error)

NewRouter creates a newt router suprisingly

func (*Router) Check added in v0.0.9 

func (rtr *Router) Check(buf io.Writer) bool

Check will review the router and its routes and return true if no errors found and false otherwise. It will write error descriptions to buf.

func (*Router) GetRouteIds added in v0.0.9 

func (rtr *Router) GetRouteIds() []string

GetRouteIds returns a list of route ids defined in the Router.

func (*Router) ListenAndServe added in v0.0.9 

func (rtr *Router) ListenAndServe() error

ListenAndServe() runs the router web service

type Service 

type Service struct {
	// AppName holds the name of the application, e.g. Postgres, PostgREST
	AppName string `josn:"name,omitempty" yaml:"name,omitempty"`

	// AppPath holds the path to the binary application, e.g. PostgREST
	// This property provides the location of the service to run.
	AppPath string `json:"path,omitempty" yaml:"path,omitempty"`

	// ConfPath holds teh path to the configuration file (e.g. PostgREST configuration file)
	ConfPath string `json:"conf_path,omitempty" yaml:"conf_path,omitempty"`

	// Namespace holds the Postgres Schema name It is used to generate
	// a setup.sql file using the -pg-setup option in newt cli.
	Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`

	// CName is the name of the dataset collection you wish to use/generate.
	CName string `json:"c_name,omitempty" yaml:"c_name,omitempty"`

	// Port is the name of the localhost port Newt will listen on.
	Port int `json:"port,omitempty" yaml:"port,omitempty"`

	// Timeout is a duration, it is used to set timeouts and the application.
	Timeout time.Duration `json:"timeout,omitempty" yaml:"timeout,omitempty"`

	// Htdocs holds any static files you want to make available through
	// Newt router.
	Htdocs string `json:"htdocs,omitempty" yaml:"htdocs,omitempty"`

	// BaseDir is used by Handlebars, usually holds the "views" directory.
	BaseDir string `json:"base_dir,omitempty" yaml:"base_dir,omitempty"`

	// ExtName is used by Handlebars to set the expected extension (e.g. ".hbs")
	ExtName string `json:"ext_name,omitempty" yaml:"ext_name,omitempty"`

	// PartialsDir is used by Handlebars to find partial templates, usually inside the views directory
	PartialsDir string `json:"partials_dir,omitempty" yaml:"partials_dir,omitempty"`

	// DSN, data ast name is a URI connection string
	DSN string `json:"dsn,omitemity" yaml:"dsn,omitempty"`

	// Environment holds a list of OS environment variables that can be made
	// available to the web services.
	Environment []string `json:"environment,omitempty" yaml:"enviroment,omitempty"`

	// Options is a map of name to string values, it is where the
	// the environment variable valuess are stored.
	Options map[string]interface{} `json:"options,omitempty" yaml:"options,omitempty"`
}

Service implements runtime config for Newt and off the shelf programs used to compose your Newt based application.

func NewServices added in v0.0.9 

func NewServices() []*Service

NewServices generates a default set of applications for your Newt project.

type Template added in v0.0.9 

type Template struct {
	// Id ties a set of one or more template together, e.g. a web form and its response
	Id string `json:"id,required" yaml:"id,omitempty"`

	// Description describes the purpose of the tempalte mapping. It is used to debug Newt YAML files.
	Description string `json:"description,omitempty" yaml:"description,omitempty"`

	// Pattern holds a request path, e.g. `/blog_post`. NOTE: the method is ignored. A POST
	// is presumed to hold data that will be processed by the template engine. A GET retrieves the
	// unresolved template.
	Pattern string `json:"request,required" yaml:"request,omitempty"`

	// Template holds a path to the primary template (aka view) file for this route. Path can be relative
	// to the current working directory.
	Template string `json:"template,required" yaml:"template,omitempty"`

	// Debug logs more verbosely if true
	Debug bool `json:"debug,omitempty" yaml:"debug,omitempty"`

	// Document hold the a map of values passed into it from the Newt YAML file in the applications
	// property. These are a way to map in environment or application wide values. These are exposed in
	// the Newt template engine `options`.
	Document map[string]interface{} `json:"document,omitempty" yaml:"document,omitempty"`

	// Vars holds the names of any variables expressed in the pattern, these an be used to replace elements of
	// the output object.
	Vars []string `json:"-" yaml:"-"`

	// Body holds a map of data to process with the template
	Body map[string]interface{} `json:"-" yaml:"-"`

	// Tmpl points to the compied template
	Tmpl *raymond.Template `json:"-" yaml:"-"`

	// BaseDir is used by holds the "views" directory.
	BaseDir string `json:"-" yaml:"-"`

	// ExtName is used by set the expected extension (e.g. ".hbs")
	ExtName string `json:"-" yaml:"-"`

	// Partials holds partials directory
	PartialsDir string `json:"-" yaml:"-"`
}

Template hold the request to template mapping for in the TemplateEngine

func (*Template) Check added in v0.0.9 

func (tmpl *Template) Check(buf io.Writer) bool

Check evaluates the *Template and outputs finding. Returns true of no error, false if errors found

func (*Template) Handler added in v0.0.9 

func (t *Template) Handler(w http.ResponseWriter, r *http.Request)

Handler decodes a the request body and then processes that as a template engine.

func (*Template) ResolvePath added in v0.0.9 

func (t *Template) ResolvePath() error

ResolvePath reviews the `.Request` attribute and updates the Vars using PatternKeys()

func (*Template) ResolveTemplate added in v0.0.9 

func (t *Template) ResolveTemplate() error

ResolvesTemplate is responsible for reading and parse the template and partials associated with a mapped request. If an error is encountered a error value is returned.

type TemplateEngine added in v0.0.9 

type TemplateEngine struct {
	// Port is the name of the localhost port Newt will listen on.
	Port int `json:"port,omitempty" yaml:"port,omitempty"`

	// BaseDir is holds the "views" for that are formed from the templates.
	BaseDir string `json:"base_dir,omitempty" yaml:"base_dir,omitempty"`

	// ExtName is used to set the expected extension (e.g. ".hbs")
	ExtName string `json:"ext_name,omitempty" yaml:"ext_name,omitempty"`

	// PartialsDir is used to find partial templates, usually inside the views directory
	PartialsDir string `json:"partials_dir,omitempty" yaml:"partials_dir,omitempty"`

	// Timeout is a duration, it is used to set timeouts and the application.
	Timeout time.Duration `json:"timeout,omitempty" yaml:"timeout,omitempty"`

	// Templates defined for the service
	Templates []*Template `json:"templates,omitempty" yaml:"templates,omitempty"`
}

TemplateEngine defines the `nte` application YAML file. It joins some of the Service struct with an array of templates so that "check" can validate the YAML.

func NewTemplateEngine added in v0.0.9 

func NewTemplateEngine(ast *AST) (*TemplateEngine, error)

NewTemplateEngine create a new TemplateEngine struct. If a filename is provided it reads the file and sets things up accordingly.

func (*TemplateEngine) Check added in v0.0.9 

func (tEng *TemplateEngine) Check(buf io.Writer) bool

Check makes sure the TemplateEngine struct is populated

func (*TemplateEngine) ListenAndServe added in v0.0.9 

func (te *TemplateEngine) ListenAndServe() error

Source Files

View all Source files

Directories

Path Synopsis
cmd
ndr command
newt command
nte command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.