Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pgsql encoder #21

Open
wants to merge 28 commits into
base: pgsql-encoder
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
15bdcb8
removed dynColSqlMaria function, simply json_encode($attributes) in e…
Jan 27, 2017
a6c08b1
runned tests for pg and updated results in pgsql1.txt
Feb 2, 2017
8922cc1
some cleanup
Feb 3, 2017
49b6dce
modified test fixtures to not set id and let pg handle it through ser…
Feb 3, 2017
415c110
modified test fixtures to not set id and let pg handle it through ser…
Feb 3, 2017
270b23c
altered pgsql encoder to create attribute the way that seems proper
Feb 4, 2017
90791cd
altered pgsql encoder to create attribute the way that seems proper, …
Feb 4, 2017
1cc09c4
fixed error of $type declared as a const
Feb 5, 2017
54f9b57
fixed error of $type declared as a const
Feb 5, 2017
e0c56b3
altered encoderInterface no reason to set $type default in interface …
Feb 5, 2017
b07f8bd
altered encoderInterface no reason to set $type default in interface …
Feb 6, 2017
4583cd6
altered encoderInterface no reason to set $type default in interface …
Feb 6, 2017
bec2b6e
updated test results
Feb 6, 2017
99d1880
updated test file
Feb 6, 2017
7742a49
updated test file altered pgencoder to use jsonb_extract_path_text wh…
Feb 10, 2017
b9ebbc6
updated test file altered pgencoder to use jsonb_extract_path_text wh…
Feb 10, 2017
0799cbb
added numeric and text types to daq, modifies pgsqlencoder to accept …
Feb 11, 2017
d5e4267
added numeric and text types to daq, modifies pgsqlencoder to accept …
Feb 11, 2017
e3e3f37
reformatted the code to comply with psr2 standards altered encoder to…
gvasilopulos Feb 11, 2017
817f1b8
created distinct regexp for postgres
gvasilopulos Feb 12, 2017
5fc39ea
added exception for not supported jsonb dbtypes under postgres
gvasilopulos Feb 12, 2017
aed81d9
bugfixes, updated test file to be "closer" to the original dynamicrec…
gvasilopulos Feb 12, 2017
7cdeb47
reformated sources to psr2 standard
gvasilopulos Feb 12, 2017
3cd8844
added boolean type
gvasilopulos Feb 13, 2017
6cd0565
added boolean type
gvasilopulos Feb 13, 2017
e4003a9
removed unused files
gvasilopulos Feb 13, 2017
31f5f7b
added test for complex json stucture, added fixtures for complex json
gvasilopulos Feb 20, 2017
768895d
added test for complex json stucture, added fixtures for complex json
gvasilopulos Feb 20, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion DynamicActiveQuery.php
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ public function createCommand($db = null)
% (`?) \(! \s*?
( [a-z_\x7f-\xff][a-z0-9_\x7f-\xff]* (?: \. [^.|\s]+)* )
(?: \| (binary (?:\(\d+\))? | char (?:\(\d+\))? | time (?:\(\d+\))? | datetime (?:\(\d+\))? | date
| decimal (?:\(\d\d?(?:,\d\d?)?\))? | double (?:\(\d\d?,\d\d?\))?
| decimal (?:\(\d\d?(?:,\d\d?)?\))? | double (?:\(\d\d?,\d\d?\))? | numeric? | text ? |
| int(eger)? | (?:un)? signed (?:\s+int(eger)?)?) )?
\s*? !\) \1 %ix
REGEXP;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you want to use the same regex for pgsql and maria then we must test that all acceptable patterns behave properly. e.g. do things like decimal, datetime and double work in pgsql? do numeric and text work in maria?

if not then the exception should make clear what the problem is.

Copy link
Author

@gvasilopulos gvasilopulos Feb 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking to implement just 3 types for pg numeric, jsonb, and char/text which is the default.
Postgres may have a hole lot of other types but in my mind that does not matter since we get json types back.
If anyone wants to typecast to anything else he can use (!dyncol|jsonb!)::postgres_data_type.
I'm thinking of including that possibility to the documentation. In fact the type declaration with the format dyncol|type was done for compatibility reasons one could use ::type straightforward in the start of the implementation.
You are right about removing it from maria's regexp. I will remove it asap and implement a regexp for pg.
I used a psr2 formatter to format the code to psr2 which I think is the yii2 coding standard.Hope it is ok now
I also added my real name and email to the git configuration

I'll get back to you with bug report later today.

Expand Down
6 changes: 3 additions & 3 deletions DynamicActiveRecord.php
Original file line number Diff line number Diff line change
Expand Up @@ -154,9 +154,9 @@ public function getAttribute($name)
return null;
}
$ref = &$ref[$key];
}

return $ref;
}
return $ref;
}

/**
Expand Down
513 changes: 291 additions & 222 deletions composer.lock

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion encoder/EncoderInterface.php
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ interface EncoderInterface
*
* @return string a SQL expression
*/
public function dynamicAttributeExpression($name, $type = 'char');
public function dynamicAttributeExpression($name, $type);

/**
* Generates an SQL expression to select value of the dynamic column.
Expand Down
149 changes: 132 additions & 17 deletions encoder/PgsqlEncoder.php
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,34 @@ class PgsqlEncoder extends BaseEncoder
*
* @return string a SQL expression
*/
public function dynamicAttributeExpression($name, $type = 'char')
public function dynamicAttributeExpression($name, $type=null)
{
$modelClass = $this->modelClass;
$sql = '[[' . $modelClass::dynamicColumn() . ']]';
foreach (explode('.', $name) as $column) {
$sql = "($sql->'$column' AS $type)";
$sqlarray= explode('|', $name);
// $sql=$sqlarray[0];
if (isset($sqlarray[1]))
{
$type=$sqlarray[1];
}

return $sql;
// $sql = '[[' . $modelClass::dynamicColumn() . ']]';
$sql=str_replace(".","','",$sqlarray[0]);
if (!isset($type))
{ return 'jsonb_extract_path('.$modelClass::dynamicColumn().',\''.$sql.'\')::jsonb';}
else if ($type=='text' || $type=='char')
{ return 'jsonb_extract_path_text('.$modelClass::dynamicColumn().',\''.$sql.'\')';}
else
{ return 'jsonb_extract_path('.$modelClass::dynamicColumn().',\''.$sql.'\')::jsonb::text::'.$type;}
}
// public function dynamicAttributeExpression($name, $type = 'char')
// {
// $modelClass = $this->modelClass;
// $sql = '[[' . $modelClass::dynamicColumn() . ']]';
// foreach (explode('.', $name) as $column) {
// $sql = "($sql->'$column' AS $type)";
// }
//
// return $sql;
// }

/**
* Generates an SQL expression to select value of the dynamic column.
Expand All @@ -44,7 +62,7 @@ public function dynamicColumnExpression()
/**
* Creates a dynamic column SQL expression representing the given attributes.
*
* @param array $attributes the dynamic attributes, which may be nested
* @param array $attributes the dynamic attributes, which may be json encoded
*
* @return null|\yii\db\Expression
*/
Expand All @@ -56,10 +74,12 @@ public function encodeDynamicColumn($attributes) {
$params = [];

// todo For now we only have Maria. Add PgSQL and generic JSON.
static::encodeDynamicAttributeArray($attributes);
$sql = static::dynColSqlMaria($attributes, $params);

return new \yii\db\Expression($sql, $params);
static::encodeDynamicAttributeArray($attributes);
$sql = json_encode($attributes); //simply encode attributes
// $sql = static::dynColSqlMaria($attributes, $params);
$sql='\''.$sql.'\''; //simply add ' ' before and after so pg accepts the value
return new \yii\db\Expression('(select CAST ('.$sql.' AS JSONB))', $params);

}

/**
Expand Down Expand Up @@ -92,10 +112,10 @@ function ($matches) {

return $decoded;
}

/**
* Creates the SQL and parameter bindings for setting dynamic attributes
* in a DB record as Dynamic Columns in Maria.
* in a DB record as Dynamic Columns in Postgres.
*
* @param array $attrs the dynamic attributes, which may be nested
* @param array $params expression parameters for binding, passed by reference
Expand All @@ -113,12 +133,10 @@ private static function dynColSqlMaria(array $attrs, & $params)
if ($value === [] || $value === null) {
continue;
}

$phKey = static::placeholder();
$phValue = static::placeholder();
$sql[] = $phKey;
$params[$phKey] = $key;

if ($value instanceof ValueExpression || is_float($value)) {
$sql[] = $value;
} elseif (is_scalar($value)) {
Expand All @@ -128,7 +146,104 @@ private static function dynColSqlMaria(array $attrs, & $params)
$sql[] = static::dynColSqlMaria($value, $params);
}
}

return $sql === [] ? 'null' : 'json_build_object(' . implode(',', $sql) . ')::jsonb';
return $sql === [] ? 'null' : 'json_build_object(' . implode(',', $sql) . ')::jsonb';
// return $sql === [] ? 'null' : '[' . implode(',', $sql) .']';
// return json_encode($sql);
}
/**
* Creates the SQL and parameter bindings for setting dynamic attributes
* in a DB record as Dynamic Columns in Maria.
*
* @param array $attrs the dynamic attributes, which may be nested
* @param array $params expression parameters for binding, passed by reference
*
* @return string SQL for a DB Expression
* @throws \yii\base\Exception
*/
// private static function jsonbColSqlPg(array $attrs, & $params)
// {
// $sql = [];
// $sql[]=$attrs;
//
// foreach ($attrs as $value)
// {
// if (is_array($value))
// {
// foreach($value as $val)
// { do
// if (is_array($val))
// {
// $val=array_shift($val);
// $sql[]=json_encode($val);
// }
// else {$sql[]=$val;}
// while (is_array($val));
// }
//
// }
//
// else
// {
// $sql[]=$value;
// }
// }

// if (is_object($value) && !($value instanceof ValueExpression)) {
// $value = method_exists($value, 'toArray') ? $value->toArray() : (array) $value;
// }
// if ($value === [] || $value === null) {
// continue;
// }
//
//// $phKey = static::placeholder();
//// $phValue = static::placeholder();
//// $sql[] = $phKey;
//// $params[$phKey] = $key;
//
// if ($value instanceof ValueExpression || is_float($value)) {
// $sql[] = $value;
// } elseif (is_scalar($value)) {
// $sql[] = $phValue;
// $params[$phValue] = $value;
// } elseif (is_array($value)) {
// $sql[] = static::dynColSqlMaria($value, $params);
// }
// }
// $sql= json_encode($sql);
// $sql= json_encode(array_values($sql));
// $sqlval=new \stdClass();
// $sql= array_shift($sql);
// $sql= array_shift($sql);
// $sqlval=(object) array($sql);
// $sqlexpr= json_encode($sqlval);
// return $sql === [] ? 'null' : $sqlexpr;
// }
// private static function dynColSqlMaria(array $attrs, & $params)
// {
// $sql = [];
// foreach ($attrs as $key => $value) {
// if (is_object($value) && !($value instanceof ValueExpression)) {
// $value = method_exists($value, 'toArray') ? $value->toArray() : (array) $value;
// }
// if ($value === [] || $value === null) {
// continue;
// }
//
// $phKey = static::placeholder();
// $phValue = static::placeholder();
// $sql[] = $phKey;
// $params[$phKey] = $key;
//
// if ($value instanceof ValueExpression || is_float($value)) {
// $sql[] = $value;
// } elseif (is_scalar($value)) {
// $sql[] = $phValue;
// $params[$phValue] = $value;
// } elseif (is_array($value)) {
// $sql[] = static::dynColSqlMaria($value, $params);
// }
// }
//
// return $sql === [] ? 'null' : 'json_build_object(' . implode(',', $sql) . ')::jsonb';
// }
}
Loading